CI: Add publishing to PyPI and TestPyPI with trusted publishers #51
Conversation
|
@asmeurer this is mostly ready to go, but before this gets finished I'll need you to add a trusted publisher to TestPyPI and PyPI with repo name https://docs.pypi.org/trusted-publishers/adding-a-publisher/ |
| # branches: | ||
| # - main | ||
| # tags: | ||
| # - v* |
There was a problem hiding this comment.
This was just commented out for me to verify things worked on my fork. I'll finish this soon and clean up and add context in PR body.
I've done this. Should I add something for the environment name? |
Ah yeah, sorry I forgot to mention this. @asmeurer can you go into https://github.com/data-apis/array-api-compat/settings/environments and add an environment named
|
matthewfeickert
force-pushed
the
ci/add-trusted-publishers-workflow
branch
from
54d49b7 to
10e7d95
Compare
There was a problem hiding this comment.
This dependabot config is only for updating GitHub Actions, and so the number of PRs generated should be substantially less than if Python packages were being considered.
| - "github-actions" | ||
| - "dependencies" | ||
| reviewers: | ||
| - "asmeurer" |
There was a problem hiding this comment.
This assumes @asmeurer wants to be the person to get the ping.
| pull_request: | ||
| branches: | ||
| - main | ||
| - release/v* |
There was a problem hiding this comment.
This is a assuming a release branch pattern for future backports for bug fixes to patch versions of release/v* which may not be the case. This can be altered as desired.
There was a problem hiding this comment.
I don't think we need to worry about backports here. I'm trying to think of a situation where we'd want to do that.
We generally try to avoid making backwards incompatible changes to the standard itself, but even if there was one, I think we would want to support multiple versions in the same array-api-compat version using api_version. Similarly, we'll probably want to support both NumPy 2.0 and NumPy 1.25 at the same time when that happens.
There was a problem hiding this comment.
I don't think we need to worry about backports here.
Sounds good. I'll drop this then if you're not planning on making releases from any branch other than main.
There was a problem hiding this comment.
This has now been removed.
| release: | ||
| types: [published] |
There was a problem hiding this comment.
This bit is for publishing on GitHub Release.
| workflow_dispatch: | ||
| inputs: | ||
| publish: | ||
| type: choice | ||
| description: 'Publish to TestPyPI?' | ||
| options: | ||
| - false | ||
| - true |
There was a problem hiding this comment.
This would allow you to create a release to TestPyPI from workflow dispatch. If this isn't of interest this can get removed.
There was a problem hiding this comment.
This is worth doing at least the first time around. How do you do it? Is there some option when you create the release in the GitHub interface?
There was a problem hiding this comment.
You do this through workflow dispatch by visiting the GitHub Actions tab for the repo and then selecting this workflow (https://github.com/data-apis/array-api-compat/actions/workflows/publish-package.yml) and then when you click the "Run workflow" button on the right hand side there will be the option to select false or true from a dropdown.
This is how it looks like for us on pyhf's GitHub repo:
and if it helps, starting around 52 seconds into this video of me making the pyhf v0.7.1 release I show this button example: https://youtu.be/ZV20tr3EpTw?t=52
|
|
||
| - name: Install python-build and twine | ||
| run: | | ||
| python -m pip install --upgrade pip setuptools |
There was a problem hiding this comment.
If array-api-compat ever moves off setuptools this setuptools bit here could be removed.
| - name: Publish distribution 📦 to Test PyPI | ||
| # Publish to TestPyPI on tag events of if manually triggered | ||
| # Compare to 'true' string as booleans get turned into strings in the console | ||
| if: >- | ||
| (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) | ||
| || (github.event_name == 'workflow_dispatch' && github.event.inputs.publish == 'true') | ||
| uses: pypa/[email protected] | ||
| with: | ||
| repository-url: https://test.pypi.org/legacy/ | ||
| print-hash: true |
There was a problem hiding this comment.
Publishing to TestPyPI (which is optional of course) would require following these instructions as there is no array-api-compat yet on TestPyPI: https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/
There was a problem hiding this comment.
Is this the right link? I don't see testpypi mentioned on that page.
There was a problem hiding this comment.
That is the right link. Both PyPI and TestPyPI have the setup for trusted publishing in the same area. So just treat all instances of PyPI as TestPyPI for creating an project with a Trusted Publishers workflow.
There was a problem hiding this comment.
bumped this from pypa/[email protected] to pypa/[email protected].
| print-hash: true | ||
|
|
||
| - name: Publish distribution 📦 to PyPI | ||
| if: github.event_name == 'release' && github.event.action == 'published' |
There was a problem hiding this comment.
This is assuming that you only want to publish to PyPI via creating a GitHub Release.
matthewfeickert
force-pushed
the
ci/add-trusted-publishers-workflow
branch
2 times, most recently
from
0502792 to
32f9ec1
Compare
|
Sorry for the 2 week delay — I was traveling for work and then finishing faculty applications — but this should be ready for review again. |
|
Thanks for the update. It might be another 2 weeks before I myself am able to look at this. |
|
No worries! Just ping whenever you have questions. |
* Use the OpenID Connect (OIDC) standard to publish to PyPI and TestPyPI using PyPI's "Trusted Publisher" implementation to publish without using API tokens stored as GitHub Actions secrets. Use an optional GitHub Actions environment to further restrict publishing to selected branches for additional security. - c.f. https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/ - c.f. https://docs.pypi.org/trusted-publishers/
* Add Dependabot config for weekly updates to GitHub Actions for security best practices. - c.f. https://learn.scientific-python.org/development/guides/gha-basic/#updating
matthewfeickert
force-pushed
the
ci/add-trusted-publishers-workflow
branch
from
32f9ec1 to
1f146b6
Compare
|
We need to do a release so I'm going to merge this and see if it works. |
@asmeurer let me know how this goes. Tag based deployments under environments have started breaking recently given https://github.com/orgs/community/discussions/62991, so for the most recent release I've made (scikit-hep/pyhf#2319) I had to temorarily set the deployment branches in the "publish-package" environment to "all branches", which invalidates #51 (comment) a bit. |
| # Publish to TestPyPI on tag events of if manually triggered | ||
| # Compare to 'true' string as booleans get turned into strings in the console | ||
| if: >- | ||
| (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) |
There was a problem hiding this comment.
@asmeurer Also catching now that as you use tags that match
- [0-9]+.[0-9]+
- [0-9]+.[0-9]+.[0-9]+
but not v* that these will never publish to TestPyPI on tag. I should fix this in a follow up PR.
Resolves #50
Use the OpenID Connect (OIDC) standard to publish to PyPI and TestPyPI using PyPI's "Trusted Publisher" implementation to publish without using API tokens stored as GitHub Actions secrets. Use an optional GitHub Actions environment to further restrict publishing to selected branches for additional security.